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PATB9310PCT 

SUBSTITUTE SPECIFICATION 
Communication terminal handling messages including graphics. 

BACKGROUND OF THE INVENTION 
Field of the Invention 

5 

The invention relates to the transmission of graphics as a part of the 
messaging between communication terminals. 

Description of the Prior Art 

10 On the Internet address http://www.kessler-design.com there is provided a 
graphic editor far generating a graphic picture on a Personal Computer in a 
predetermined formatted file, and to route this file via the Internet to a server 
where the message, based on the "Smart Messaging Concept" developed by 
the applicant is generated based on the received file. From here the message 

15 is routed to the cellular phone via an SMS gateway & SMS interworidng 
Mobile Switching Center (MSC). 

Philips Consumer Communications has exposed a new phone SAWY™ 
GSM phone in Cannes 23. February 1999. This phone will be equipped with 

20 a message application using graphics. It is possible to include "clip art" like 
graphics in messages exchanged between two SAWY™ phones. Basically 
this means that the "clip art" like graphics are handled as predefined 
characters and the character is transformed to a hexadecimal code in one 
phone, transferred back to the same predefined character in the receiving 

25 phone. 

SUMMARY OF THE INVENTION 
The invention provides a method for handling transmission of messages 
including graphics between communication terminals. 

30 

This purpose is obtained by a method for handling messages transmitted 
between communication terminals via a wireless network, and comprising 



PAGE 16/39 ' RCVDAT 11/24/20042:03:57 PM [Eastern Standard rime] ^ SVR:USPtO-EFXRF-1/1 * DNIS;8729309 * CSID:7033 126666 * DURATION (mm-ss):10-26 



11/24/2084 14:05 7033126666 



BR I NT 



PAGE 17/39 



2 

generation of a compound message including a text part and at least one 
graphical icon part and transmission of the message via the wireless network. 
The handling of the compound message generation comprises steps of 
reading of a user inputted text part and converting the inputted text into in a 
5 predefined message text format, adding a graphical part to the message, said 
graphical part including a record for each of said at least one graphical icon 
part in a graphical format, and adding information in the message defining the 
position of said at least one graphical icon part in the text part. Hereby the 
phones will be able to include a real graphical file into the message and the 
io receiving phone will be able to decode the graphical Icon without knowing the 
pattern in advance. 

The invention furthermore relates to a communication terminal for handling 
messages and having a controller, a transceiver for communicating with a 

is wireless communication network, and a user interface through which the user 
operates the terminal, the user interface including a display. The 
communication terminal furthermore comprises a message editor application 
by means of which the user is allowed to generate a compound message 
including a text part and at least one graphical icon part; said controller 

20 generates a message for transmission via said transceiver. The message 
includes a text part in a predefined message text character format, a 
graphical part including a record for each of said at least one graphical icon 
part in a graphical format, and information rn the message defining the 
position of said at least one graphical icon part In the text part. The graphics 

25 used in the message may be found in a clip art like gallery including a 

plurality of graphical icons, or manually entered by the user by means of a 
graphical editor. 

The invention furthermore relates to a message format including a text part 
30 and at least one graphical Icon part. The message includes a text part in a 
predefined message text character format, a graphical part including a record 
for each of said at least one graphical icon part in a graphical format, and 
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information In the message defining the position of said at least one graphical 
icon part in the text part. Hereby an existing text message carrying concept 
such as for example SMS messaging in the GSM system is able to transport 
graphics when the receiving and transmitting terminals have respective 
5 graphical editors/readers for coding and decoding the messages. 

According to the preferred embodiment this information is handled by the 
Smart Messaging Concept. This concept was disclosed by the applicant at 
the Cebit fair in March 1997 (Press release 20. March 1997) and discussed 

io in details the document: "Smart Messaging Specification", Revision 1,0.0; 
September being available on the Internet address URL: 
www.forum.nokia.com/nf/concepts/smart/main.html. From this Internet 
address another document "Narrow-Band Sockets Specification", Revision 
1 .0, March 7, 1997 may be downloaded. This document describes the 

15 Narrow-Band Socket (developed by a joint effort by of Intel and Nokia) that 
enables applications to access various network data bearer services using a 
standard socket interface. 

According to the invention it Is very important to improve the character of 
20 sending messages in a cellular phone network from being a pure text 
messaging system to become a more sophisticated concept including 
graphics into messages and thereby attracting younger people who want to 
generate tags or symbols identifying themselves when sending messages. 

25 In addition to offering a huge amount of templates or predefined graphics the 
invention according to the preferred embodiment offers the user a built-in 
graphics composer application by means of which the user is allowed to 
create graphics directly on the phone. The user wfll also not need to buy a PC 
interface cable, to be able to create graphics on a PC. 
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4 

According to the invention it is essential to offer sending and receiving 
graphics directly from one phone to another Furthermore the graphics are 
actually included into the message text. 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

For a better understanding of the present invention and to understand how 
the same may be brought into effect reference will now be made, by way of 
example only, to accompanying drawings, in which:- 

io Fig. 1 schematically illustrates a preferred embodiment of a hand portable 
phone according to the invention* 

Fig, 2 schematically shows the essential parts of a telephone for 
communication with a cellular or cordless network. 

15 

Fig. 3 illustrates the message reception scenario according to the invention. 

Fjg. 4 illustrates the message transmission scenario according to the 
Invention. 

20 

Fig. 5 shows a display sequence of a phone receiving a message including 
graphics according to the Invention. 

Fig. 6 shows a display sequence of a phone inserting graphics into a 
25 message for transmission according to the invention. 

Fig. 7 shows a display of a graphical editor of a phone for manually entering a 
new graphical icon according to the invention. 

30 Fig. 8 schematically shows the essential parts of a GMS editor/reader 
according to the invention. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 
Fig. 1 shows a preferred embodiment of a phone according to the invention, 
and it will be seen that the phone, which is generally designated by 1 r 
comprises a user interface having a keypad 2, a display 3 t an on/off button 4, 
s a speaker 5, and a microphone 6 (only openings are shown). The phone 1 
according to the preferred embodiment is adapted for communication via a 
cellular network, but could have been designed for a cordless network as 
well. 

io According to the preferred embodiment the keypad 2 has a first group 7 of 
keys as alphanumeric keys, a soft key 8, a clear key 9, and a navigation key 
or a scroll key 1 0. The present functionality of the soft key 8 is shown in a 
separate field in the display 3 just above the key 8. 

15 Fig. 2 schematically shows the most important parts of a preferred 

embodiment of the phone, these parts being essential to the understanding of 
the invention. The preferred embodiment of the phone of the invention is 
adapted for use in connection with the GSM 900MHz and GSM 1 800 MHz 
network, but, of course, the invention may also be applied in connection with 

20 other phone networks. The processor 18 controls the communication with the 
network via the transmitter/receiver circuit 19 and an antenna 20 that will be 
discussed in details below. 

The microphone 6 transforms the user's speech into the analog signals, the 
25 signals formed thereby are A/D converted in an A/D converter (not shown) 
before the speech is encoded in an audio part 14. The encoded speech 
signal is transferred to the processor 18, which is supports the GSM terminal 
software. The processor 18 also forms the interface to the peripheral units of 
the apparatus, including a RAM memory 17a and a Flash ROM memory 17b, 
30 a SIM card 1 6 r the display 3 and the keypad 2 (as well as data, power supply, 
etc.). The audio part 14 speech-decodes the signal, which is transferred from 
the processor 18 to the earpiece 5 via a D/A converter (not shown). 
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Receiving messages including graphics. 

The abiirty to receive and send graphics via messages will require the 
features of storing a targe amount of pre-specified graphics inside the phone 

5 (Clip-art / graphics templates}* the ability to insert graphics into a message 
being composed, a method to browse the huge amount of graphics and a 
possibility to save received graphics, and to afterwards use them in message 
composing. According to the preferred embodiment of the invention it is 
furthermore possible to generate new graphics on the wireless 

10 communication terminal or the phone 1 . 

According to the preferred embodiment when the user receives a message, 
the reception will be indicated in Idle mode of the phone 1, like the first image 
of Fig. 5. A header 50 includes some icons informing the user about for 

is example the status of phone settings, time and un-read messages present in 
the phonebook. Furthermore it is seen from the first line that a new message 
is received, and the second line indicates the phone number or the name of 
the sender If this can be found in the phonebook based on the phone number 
of the sender. The next two lines indicate the start of the message. If several 

20 messages are received, the initial text from the newest message will be 

displayed. The functionality of the soft key 6 will advantageously be "read" as 
seen from the first display. Pressing this softkey 8 allows the user to read and 
edit the entire message as shown In the second display ofFig. 5. 

25 In this case, the user has chosen to read the message, simply by pressing 
the soft-key 8 having the "Read" functionality. This will directly enter the 
message viewing as seen in the second display of Fig. 5. Now the 
functionality of the soft key 8 changes to "option", and by pressing this key 8 
the user gets direct access to e.g. delete, edit or forward the message. By 

30 using the scroll key 10 the user may scroll through the message as seen in 
the third display from the second and third display it is seen that the header 
50 now identifies the sender of the message. Furthermore it is seen that the 
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message includes graphics 51 formed as two Dolphins (the trademark of the 
Danish operator Sonofon). 

Graphics can basically be of two types: "Large graphics" - as the Dolphins 
5 shown in the third display; and "In-line graphics" that means graphics with 
maximum height of that corresponding to the height of the text, e.g. 8 pixels. 
"In-line graphics" are placed in a text line and displayed as a text character. 
However the handling of the In-line graphics in the message will be similar to 
the handling of Large graphics. 

10 

The fourth display shows a message including "In-line graphics" 52. The 
width of the graphics can basically be up to the width of a text line. However a 
maximum width of two-three characters makes it easier to handle the 
message display. If the user (in the second, third or fourth display) now wants 
is to save the graphics the soft key 8 is pressed havfng the "Options" 

functionality and select "user graphics" in the option list displayed in the fifth 
display. 

When the "User Graphics" option is selected a list (page wise) of all the 
20 graphics stored in the message will be given. In the message shown in Fig. 5 
second and third display only a single graphic 51 is included. The default 
options on the soft-key 8 will now allow the user to store the graphics directly 
to the "user graphics" gallery of the phone 1 for later use of the graphics. 

is Fig. 3 illustrates the message received scenario. In step 100 a message Is 
received and in step 101 the GRAPHICAL MESSAGING SYSTEM (GSM) 
reader 30 checks whether the received message is in the GMS format or not. 
If not the GMS reader 30 just starts to wait for the next message. Otherwise 
the GMS reader 30 reads the message and starts to identify the text part and 

30 possible graphics in step 102. When this is done the GMS message is 

displayed for the user in step 103. [n step 104 the user is allowed to store the 
received graphics and this is done In step 105 upon request from the user. In 



PAGE 2209 ' RCVD AT 11/24/2004 2:03:57 PM [Eastern Standard Time] » SVR:USPT0<EFXRF-1/1 ■ DNIS:8729309 ' CSID:7033126666* DURATION (mm-ss):10-26 



11/24/2004 



14:05 7033126666 



BRINT 



PAGE 23/39 



8 

step 106 the display of the messages remains until the user wants to close 
the message. 

InFig. 4 it is shown that the GMS editor starts in step 110 when the user 
5 wants to enter a GMS message. In step 111 the entered text is read and 
when the user wants to enter graphics the user can indicate this in step 112. 
Then the user will have the choice (in step 1 13) of entering a new graphic by 
means of using the graphical editor 35 (step 114) of by recalling a graphical 
icon stored in one of the memories 32 or 33 (fig. 8) in step 115. When the 
i o user does not want to enter further text or graphics the user can request 
transmission in step 116 whereby the GMS message is transmitted in step 
117 and the application is closed. In stepl 16 the user will be allowed to close 
the application without transmitting the message. 

15 Sanding messages including graphics. 

The user has the ability to insert graphics into the message. The user can 
insert graphics from both a pre-defined Clip-art directory, and from the "user 
graphics" gallery. In the first display of Fig. 6 the user has started to compose 
a message. The soft-key 8 now allows the user directly to send the message, 
20 but by using the scroll key 10 alternative functionalities, including insert of 
graphics in the option called "Insert Graphic" as seen in the second picture of 
Fig. 6. By selecting the "Insert Graphics" option the user will be given a list of 
graphics groups, like: User Graphics, Animals, Business, Cartoons etc. as 
shown in the third display of Fig. 6. 

25 

This list will contain logical groups of graphics from the clip-art database, and 
- as the first item - the complete list of the graphics stored by the user. The 
list will also contain an item called "In-line icons", which will contain icons that 
can be included in the text lines, preferably having a maximum height 
30 corresponding to 6 pixels. Preferably, the list will also contain CLI icons, start- 
up graphics etc. Selecting any of the items in this list, will give the user a list 
of all the graphics in the group. Selecting for example the item called 
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"Animals 0 could give one of the graphics shown in the fourth or fifth display of 
Fig. 6. Scrolling down with the scroll key 10 will show the next graphic in the 
"Animals 0 group. Pressing the soft-key 8 having the functionality "Insert", will 
insert the graphic into the message text at the position of the cursor 53 (first 
5 display of Fig. 6). The graphic is now inserted into the text, and the editor is 
ready for insertion of a new word to be added after the graphic as shown in 
the sixth display of Fig. 6. 

According to the invention it is very important to improve the character of 
10 sending messages in a cellular phone network from being a pure text 

messaging system to become more sophisticated to permit including graphics 
into messages and thereby attracting younger people who want to generate 
tags or symbols identifying themselves when sending messages. 

15 The graphics editor basically allows the user to edit and define any pixels in 
the display. The pixel editor display is displayed in Fig. 7 and each pixel in 
the final drawing consists of 3 x 3 pixels in the editor. The current position of 
the cursor is indicated with a hollow pixel 70, A small preview 71 of the final 
drawing can be shown in the lower right comer of the display. By using the 

20 soft key functionality "Options", the user can for example invert a specific 
pixel. The basic soft key functionalities under "Option" could allow the user to 
simply invert pixels individually, but functions like "Circle", "Insert Text", 
"Reverse all" will also be included. The cursor or hollow pixel 70 is moved in 
the display by means of navigation means. The navigation key 10 only allows 

25 the user to navigate in two directions, and therefore it is preferred to dedicate 
the "Z\ *4\ "6" and a 8" keys for moving the cursor in the four directions of the 
display. However an improved embodiment could include a four directional 
scroll key. 

3o Animations. 

Like with messages and graphics, the graphical reader could offer the user a 
variety of pre-defined animations, which the user can attach to different 
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"events" in the user interface. These events could for example be when 
entering a menu item, start-up animations, when an incoming call appears 
(for example CLI Group dependent), when the keyguard is activated or de- 
activated, etc. 

5 

It is evident that the invention supports a scenario in which the user creates 
the user's own animations, for example on a Personal Computer and then 
downloads the animation to the phone which is used just like the pre-defined 
animations. The user can just send the animation to another user, 

10 

Basically these animations are a sequence of pre-stored graphics that are 
displayed as a sequence. An animation generator has to be able to identify 
and access the individual graphics in a memory. Furthermore a set of 
parameters has to be present for the individual animation and the parameters 

15 must contain an identification of a start event, such as start up of the phone, 
detection of an incoming call etc. Furthermore the refreshing time or period 
has to be Identified that is the time one graphic is displayed before it is 
replaced by the next graphic. Finally the duration of the animation has to be 
given and this will typically be when the full animation has been played once, 

20 or when the graphics in the full animation have been played as an endless 
sequence for example two or three times. 

Graphical Messaging System, 

The QMS is able to transport text, pictures and animations as one entity in 
25 messages handled by for example the Nokia Smart Messaging format. The 
Nokia Smart Messaging format is presently used in connection with the GSM 
messaging system and therefore the overall concept may be regarded as well 
known to a person skilled in the art. The format of the messages including 
graphics transmitted from one phone to another according to the preferred 
30 embodiment of the invention will be described below with reference to Fig. 4. 
The message format is based on the Nokia/Intel Narrow Band Socket (NBS) 
specification. This format is already well known and used for a number of 
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11 

services already implemented In a wide range of Nokia products, for example. 
Over The Air (OTA) business cards, OTA ringing tone download etc. 



A GMS reader 30 is listening to a Narrow BAND Socket (NBS) port 5514 
5 decimal [158A hexadecimal] via the processor 18. The received message is 
displayed In the display 3 and temporarily stored in a memory 34. The reader 
30 identifies the text part and the graphics part of the message and the user 
is as mentioned above, able to store the received graphics in a "User 
Graphics'* memory 32 for later user. 

10 

It is well known to use text editors in for example GSM phones for generating 
a text for a message to be transmitted. A GMS editor 31 replaces this text 
editor and acts fully as such one when handling text alone. The user is able 
to enter a text part by using the text entry means — preferably the 

is alphanumeric keys, and when the user via the "Option" functionality in the 
soft key 8 starts to enter graphics the user can get one precoded graphical 
icon from a memory 33 including a plurality of clip art like graphics - 
. preferably included in a plurality of groups (animals, business, cartoons etc.) 
in order to ease the search for a specific icon. Alternatively the user can enter 

20 the desired pattern by means of a graphical editor 35 (display is shown in Fig. 
7) and store the manually entered graphic in a "User graphic" memory 32 and 
use the graphic in the GMS editor. The "User graphic" memory 32 includes 
manually entered graphics, graphics received in a message, or clip art like 
graphics amended in the graphical editor 35. 

23 

GMS format. 

The actual GMS format will be based on the Nokia Smart Messaging format, 
but the actual format will be as follows: 

30 < GMS-message> ::=* <GMS-header>< GMS-item> + 

< GMS-header> ::= < GMS-Version> 

< GMS-Versfon> ::= 'O* j'ldentffier for GM3 version, current version is zero (0). 
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12 

<GMS-ftem> W <GMS-item-length> <lSO-8859-l-char>* 
| "01" <GMS-item-length> <UNICODE-char>* 
| "02* <GMS-item-iength> <OTA-bltmap> 

| "03" <GMS-itern-length> <Ringing-tone-programming-language> 
S | <common-hex-digft> <common-h ex-dig it> <GSM-item-length> 

<unspecified-data> 

; Two hex characters who specify the GSM item type, if not 00.. .03. 
Tokens 04 ... FF are reserved. If one of these values Is present, the 
application may either stop processing or skip that QMS-item.' 
10 <unspecffieoVdata> ::= <default-char>* ; 'A stream of octets' 

<GMS-item-1ength> ::= 
<common-hex-digft> <common-hex-digit> 
<eommon-hex-digit> <common-hex-dlgrt> 
; 'Number of octets in the following field, in hexadecimal. 
15 Note that there are two octets per each Unicode character. 1 
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